Simultaneous Navigation and Radio Mapping for Cellular-Connected UAV With Deep Reinforcement Learning

Cellular-connected unmanned aerial vehicle (UAV) is a promising technology to unlock the full potential of UAVs in future by reusing cellular base stations (BSs) enable their air-ground communications. However, how achieve ubiquitous three-dimensional (3D) communication coverage for sky new challenge. In this paper, we tackle challenge coverage-aware navigation approach, which exploits UAV's controllable mobility design its navigation/trajectory avoid BSs' holes while accomplishing missions. To end, formulate an UAV trajectory optimization problem minimize weighted sum mission completion time and expected outage duration, which, however, cannot be solved standard techniques due lack accurate tractable end-to-end model practice. overcome difficulty, propose solution approach based on technique deep reinforcement learning (DRL). Specifically, leveraging state-of-the-art dueling double Q network (dueling DDQN) with multi-step learning, first algorithm direct RL, where signal measurement at used directly train action-value function policy. further improve performance, framework called simultaneous radio mapping (SNARM), not only training DQN directly, but also create map that able predict probabilities all locations area interest. This enables generation simulated trajectories predicting returns, are then via Dyna technique, thus greatly improving efficiency.